REMARKS 

Applicants have corrected the amendment to show claims 1-90 as 

(cancelled). 

The remarks of the previous amendment are reproduced here. 
Claims 91-130 were pending. Claims 131-152 stand withdrawn. Claims 91-130 
stand rejected. Claims 92, 100, 107-120, and 126-152 were cancelled. Claims 91, 93, 101, 106, and 
121 were amended. Claims 153-172 were added. Claims 153-172 remain in the application. 

Claims 92, 100, 107-152 were cancelled. Claims 93 and 101 were amended to 
incorporate language formerly in Claims 92 and 101, respectively. 

In the Decision on Appeal, new grounds of rejection of Claims 91-105 and 121-130 
were made under 37 CFR § 41 .50(b). The Board stated: 

"We make the following new grounds of rejection using our authority under 
37 CFR § 41.50(b). 

"Claims 91-105 and 121-130 are rejected under 35 U.S.C. 112, second 
paragraph, as being incomplete for omitting essential elements, such omission amounting to a 
gap between the necessary structural connections. A claim which fails to interrelate essential 
elements of the invention as defined by applicants) in the specification may be rejected under 
35 U.S.C. 112, second paragraph, for failure to point out and distinctly claim the invention. 
See In re Venezia, 530 F.2d 956, 957-58, 189 USPQ 149 (CCPA 1976); In re Collier, 397 
F.2d 1003, 1 105, 158 USPQ 266, 267 (CCPA 1968). 

"All of Appellants method claims share the same problem in that they are 
not limited to a machine implemented method. Normally no such limitation is required. 
However, in the situation before us, Appellants have specifically stated that the invention 
"provides the fundamental advantage of eliminating the need for manual intervention." See 
Appellants' specification at lines 1-2 of page 6. Appellants' method claims are not currently so 
limited. 

"We do not read the claim language "automatically" as requiring a machine. 
For example, the "automatically identifying" in line 3 of claim 91 does not require a machine 
as it also covers a manual operation performed in a rote or mechanical fashion by a human. 
We also note that the mere complexity of the steps in Appellants* method claims does not in 
itself require that a machine perform them." 

The independent method claims overcome this rejection as follows. Claims 91 and 
121 were amended by adding: 

"wherein said steps are without manual intervention and said altering 
follows any and all identifying of said main subject." 
This language is supported by the application as filed, notably at page 21, lines 1-5 and page 5, line 27 
to page 6, line 2. Added Claim 157 similarly states: 

"wherein said identifying and altering steps are without manual 

intervention." 



and added Claim 164 states: 

"wherein said steps are without manual intervention." 
In the final rejection sustained by the Board, Claims 91-130 stood rejected under 35 
U.S.C. 102(e) as being anticipated by U.S. Patent No. 6,504,951 to Luo et al. The final rejection 
stated: 

"As to claim 91, Luo discloses a method for modifying an image having 
pixels, comprising: 

"automatically identifying a main subject (candidate sky) of the image 
(figures 6, 73-74, column 9, lines 25-35); and 

"automatically altering pixel values (eliminating pixel having tecture above 
threshold or fills voids of connected components) to emphasized the main subject, said 
altering following identifying (abstract, figures 6, 75, columns 9, lines 38-39, 57-67 note that 
thresholding, eliminating, and region growing are conducted after identifying the main 
subject, candidate sky). 

"As to claim 92, Luo further discloses the step of identifying comprises 
segmenting the image into a plurality of regions (col. 9, lines 25-43). 

"As to claim 93, Luo further discloses generating a belief map of said region 
(fig. 6 71, col. 9, lines 15-27, col. 8 lines' 60-64). 

"As to claims 98-99, Luo further discloses altering emphasizes the main 
subject are part of said main subject (abstract, fig. 6, 75, col. 9, lines 38-39, 57-67, note that 
filling voids of connected components that is main subject, and eliminating connected 
components that a not blue sky). 

"As to claims 100-101, Luo further discloses desaturating the pixels that are 
not a part of said main subject (col. 16 lines 3-14) and in desaturating further comprises: 
calculating luminance values of the pixels that are not a part of main sibject and replaying the 
color values of the pixels that are not a part of the main subject with respective luminance 
value (abstract, fig. 6, 75, col. 9 lines 38-39, 57-67, col. 15 line 65-col. 16 line 14). 

"As to claims 102-104, Luo further discloses enhance saturation of the main 
object pixels (sky) (and inverting the hue values of pixels corresponding whether the pixels 
are the part of main subject (sky) (col. 10 lines 32-67). 

"As to claim 105, Luo further discloses segmenting the image into plurality 
of regions and assigning a believe value to the pixels corresponding to the level of saliency 
(col. 9 lines 15-42). 

"As to claims 94-97, Luo further discloses altering color saturation (col. 16, 
lines 2-14), hue (color, col. 10, lines 32-67, note that adding blue to pixels in the void or 
eliminating color pixels not in sky (set to respective luminance) can be viewed as altering 
hue), luminance and pixel blur (abstract, column 10, lines 48-67, column 15, lines 26-37). 

"As to claims 106-120, the claims are corresponding system claims to 
claims 91-105. The discussions are addressed with regard to claims 91-105. 
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"As to claim 121, Luo discloses a method of modifying an image having 
pixels, comprising: 

"automatically generating one or more belief values, each being associated 
with one of plurality of regions of the image, said belief values each being related to the 
probability that the associated region is a main subject of the image (figure 2, 201, figure 4a-d, 
column 8, lines 47-64); 

"following said generating, automatically altering pixel values in the 
plurality regions of the image, in accordance with the associated belief values (fig. 6, 75, col. 
10, lines 49-67). 

"As to claim 126, the limitations are addressed with regard to claim 121. 
"As to claims 122-125 and 127-130, the limitations are addressed with 
regard to claims 94-105." 
Amended Claim 91 states: 

91. A method for modifying an image having pixels, comprising the steps 

of: 

automatically identifying a main subject of the image, and 
automatically altering pixel values of said image to emphasize said main 
subject, said altering following said identifying; 

wherein said steps are without manual intervention and said altering follows 
any and all identifying of said main subject. 
Claim 91 is supported by the application as filed, notably the original claims and at page 14, line 28 to 
page 15, line 3. 

Claim 91 requires that the altering follows any and all identifying of the main 
subject. This feature is distinct from Luo (and from the claims that were appealed), since the multi- 
stage identification process of Luo is precluded. (See Luo, col. 5, lines 2-3; col. 10, lines 5 and 49; 
and the discussion in the Decision on Appeal, paragraph beginning on page 6 and concluding on page 
7) 

Claims 92 and 100 were cancelled. 

Claims 93-99 and 101-105 are allowable as depending from Claim 91 and as follows. 
Claims 94-97 state: 

94. The method of Claim 9 1 wherein said altering pixel values further 
comprises altering pixel color saturation. 

95. The method of Claim 91 wherein said altering pixel values further 
comprises altering pixel hue. 

96. The method of Claim 91 wherein said altering pixel values further 
comprises altering pixel luminescence. 

97. The method of Claim 91 wherein said altering pixel values further 
comprises altering pixel blur. 

Unlike Claims 94-97, Luo does not disclose altering saturation, luminance, hue, and blur in an altering 
step that follows any and all identifying of a main subject. Luo instead uses features, such as color, 



brightness, and texture, in the identification of regions. (See Luo, Fig. 2; col. 7, lines 9-10; col. 7, lines 
46-47; col. 8, lines 2-8; col. 6, lines 51-57; col. 8, line 40-col. 9, line 31.) In identifying features, Luo 
produces a list of segmented regions ranked in belief order. The list can be converted into a map in 
which brightness of the regions shows belief order. (Luo, col. 15, lines 29-31 ; col. 3, line 66 to col. 4, 
line 4. The map can be output. Luo, col. 15, lines 40-44; col. 17, lines 1-3. See col. 15-17 generally.) 
The map remains part of the identifying, even if identifying is repeated. 

Claims 101-104 all relate to the altering of pixel values of one of the properties 
discussed in relation to Claims 94-97 and are allowable on the same grounds as those claims. 

Claim 106 is supported and allowable on the same grounds as Claim 91 . Claims 107- 
120 were cancelled. 

Claim 121 states: 

121 . A method of modifying an image having pixels, comprising the steps 

of: 

automatically generating one or more belief values, each said belief value 
being associated with one of a plurality of regions of the image, said belief values each being 
related to the probability that the associated region is a main subject of the image; 

following said generating, automatically altering pixel values in said 
plurality of regions of said image, in accordance with said associated belief values; 

wherein said steps are without manual intervention and said altering follows 
all said generating. 
Claim 12 1 is supported in the same manner as Claim 91. 

Claim 121 requires generating belief values that relate to the probability that an 
associated region is a main subject and that altering pixels follows "all said generating" and is 
allowable on the same grounds as Claim 91. 

Claims 122-125 are allowable as depending from Claim 121 and on grounds 
discussed above in relation to Claims 94-97. 

Claims 126-130 were cancelled. 

Added Claims 153-156 are allowable as depending from Claim 91 and as follows. 
Claim 153 states: 

153. The method of Claim 91 wherein said main subject includes skin tone 

pixels. 

Claim 153 is supported by the application as filed, notably the original claims and at page 12, line 9 to 
page 13, line 3. Claim 153 requires that the main subject defined by the computed main subject belief 
map includes skin tone pixels. Luo excludes non-sky pixels, even if similar in color. (See, for 
example, Luo, col. 15, lines 49-52) 

Claims 154-155 states: 

154. The method of claim 93 wherein said segmenting further comprises 
extracting features of said input digital image and said generating further comprises using a 
reasoning engine to compute said belief map from said features. 
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155. The method of claim 154 wherein said reasoning engine is Bayes net- 
based. 

Claims 154-155 are supported by the application as filed, notably the original claims and at page 8, 
lines 7-8 and page 9, lines 28-29. 

Claim 156 states: 

156 (new). The method of claim 154 wherein said features include semantic 
features and structural features, said semantic features including skin, said structural features 
including centrality. 

Claim 156 is supported by the application as filed, notably the original claims and at page 9, lines 21- 
23; page 12, line 9 to page 13, line 3; and page 10, line 3 to page 11, line 3. Claim 156 requires the 
extracted semantic feature of skin and the extracted structural feature of centrality. Luo excludes non- 
sky pixels even if similar in color (See, for example, Luo, col. 15, lines 49-52) and looks to traces 
extending from horizon-to-zenith. (Luo, col. 1 1, line 1 to col. 12, line 15) 
Claim 157 states: 

1 57. A digital image modification method for use on an input digital image 
to produce an output image, the method comprising the steps of: 

receiving a user selection of one of a plurality of different types of 
emphasizing to provide a selected emphasis; 

identifying a main subject and a background in the input digital image; 
altering pixel values of said input digital image, in accordance with said 
selected emphasis, to provide the output image wherein said main subject is emphasized; 

wherein said identifying and altering steps are without manual intervention. 
Claim 157 is supported by the application as filed, notably the original claims and at page 21, lines 15- 
22 and page 21, lines 27-28. 

Claim 157 requires receiving a user selection of one of a plurality of different types 
of emphasizing to provide a selected emphasis. Pixels are then altered in accordance with the selected 
emphasis. Luo lacks any such step, nor is it apparent how such a receiving step would be compatible 
with steps utilizing a physical model of the sky. 

Claims 158-163 are allowable as depending from Claim 157 and as follows. 
Claims 158-159 state: 

158. The method of claim 157 further comprising delivering said output 
image to an output device. 

159. The method of claim 158 wherein said output device is a printer or a 

display. 

Claims 158-159 is supported by the application as filed, notably at page 21 , line 28 to page 22, line 2. 
Claim 158 requires delivering an output image having pixel values altered "wherein said main subject 
is emphasized" to an output device. Claim 159 requires that the output device is a printer or a display. 
Luo does disclose these features. Figure 1 of Luo shows modification of an input image, but that 
modification is part of a larger system that uses the results of the sky detection process. Luo states: 
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"The subject matter of the present invention relates to digital image 
understanding technology , which is understood to mean technology that digitally processes a 
digital image to recognize and thereby assign useful meaning to human understandable 
objects, attributes or conditions and then to utilize the results obtained in the further 
processing of the digital image. 

"A block diagram of the overall sky detection system (e.g., the digital image 
understanding technology) is shown in FIG. 1. First, a digital image 10 is digitally processed 
20. The results 30 obtained from processing step 20 are used along with the original digital 
image 10 in an image modification step 40 to produce a modified image 50. 

"A more specific block diagram of the inventive sky detection process is 
shown in FIG. 2." (Luo, col. 4, line 55 to col. 5, line 2) 
Luo is silent as to the nature of the modification in step 40. 

Claim 160 is supported and allowable on the same grounds as Claim 153. 
Claims 164-165 state: 

1 64. A digital image modification method for use on an input digital image 
in a computer system to produce an output image, the method comprising the steps of: 

segmenting the input digital image into a plurality of regions, said regions 
each being homogeneous; 

computing a main subject belief map of said regions, said map defining a 
main subject in said input digital image; 

then, altering pixel values in said input digital image, in accordance with 
said map, to provide the output image wherein said main subject is emphasized; and 

next, delivering said output image to an output device; 

wherein said steps are without manual intervention. 

165. The method of claim 164 wherein said output device is a printer or a 

display. 

Claims 164-165 are supported and allowable on the grounds discussed in relation to Claims 158-159. 
Claim 164 is also allowable as requiring an automated step of outputting the output image to an output 
device. This feature is supported by the application as filed, notably at page 22, lines 1-2. This feature 
is not disclosed by Luo. 

Claims 165-172 are allowable as depending from Claim 164 and as above and below. 

Claim 166 states: 

166. The method of claim 164 wherein said computing of said map is based 
upon degree of centrality, borderness, and chrominance space distribution. 

Claim 166 is supported by the application as filed, notably page 10, line 3 to page 13, line 3. Claim 
166 requires that the main subject defined by the computed main subject belief map is based upon 
degree of centrality, borderness, and chrominance space distribution. This contrasts with the approach 
of Luo, which looks to traces extending from horizon-to-zenith. (Luo, col. 11, line 1 to col. 12, line 15) 

Claim 168 is supported and allowable on the same grounds as Claim 153. 

Claim 170 is supported and allowable on the same grounds as Claim 156. 
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Claims 171-172 state: 

171. The method of claim 170 wherein said main subject has a high degree 

of said centrality. 

Claim 171 is supported by the application as filed, notably at page 10, line 3 to page 11, line 3. Claim 
171 requires that the main subject has a high degree of centrality. This contrasts with Luo, in which the 
sky is detected using traces extending from horizon-to-zenith. (Luo, col. 11, line 1 to col. 12, line 15) 
Claim 172 is supported and allowable on the same grounds as Claim 153. 

It is believed that this correction along with the response to the 
Decision on Appeal mailed on October 2<5, 2005 is complete and that the claims are in 
condition for allowance. Favorable reconsideration and early passage to issue is 
therefore earnestly solicited. 
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